Spoken Dialogue System Using Prosody as Para-Linguistic Information

An attitude recognizer of a speaker which uses prosodic features of speech is proposed and it is successfully applied to the dialogue system aiming at agreement formation. We use not only linguistic information but also some sorts of additional information supporting linguistic information in our human communication. In agreement formation dialogues, we are often required to express our attitude (positive or negative) to conversational partners’ proposals. We sometimes reply explicitly in linguistic information. We sometimes reply information ambiguously. However, even in the ambiguous case, we implicitly express our attitude using prosodic information. By realizing the abilities of catching these nuances, the dialogue system can be more sophisticated. In this paper, we implemented an attitude recognizer based on the GMM using prosodic feature parameters. The performance of the system is comparable to the human ability. We also realized a proto-type of spoken dialogue system using the recognizer. We show how these abilities contribute to efficient conversation.